Real-time Hallucination Sonification and Simulation through User-led Devel- Opment of an Ipad Augmented Reality Performance

The simulation of visual hallucinations has multiple applications. For example in helping diagnosis, in helping patients to express themselves and reduce their sense of isolation, for medical education, and in legal proceedings for damages due to eye / brain injuries. We present a new approach to hallucination simulation, which was developed initially for a performance but proved to have potential uses to sufferers of certain types of hallucinations. The system allows real-time audio and visual expression, using an iPad. An individual can overlay their hallucinations in real-time on the iPad screen over the iPad’s video camera image. The system has been developed focusing on the visual symptoms of Palinopsia, experienced by the first author, and hence has initially been user-led research. However such an approach can be utilized for other conditions and visual hallucination types. The system also allows the hallucinations to be converted into sound through visual sonification, providing another avenue for expression for the hallucinating individual. A musical performance is described which uses the system, and which has helped to raise awareness and to comfort some people who have Palinopsia symptoms. Although no formal experimentation was done outside of performance preparation, we report on a number of unsolicited informal responses to the simulator from palinopsia sufferers and a palinopsia charity.